The present disclosure relates generally to the management of data streams generated by computer hardware. In particular, methods and systems enabling the sharing of a single hardware data stream with multiple consuming applications are described.
Modern mobile devices are commonly equipped with hardware such as a camera that can be used by a variety of applications capable of being executed by the mobile device, so as to perform a variety of functions. For example, applications exist that can take pictures and video captured by the camera and manipulate them, edit them, insert them into other media, and/or upload them to online services for further distribution and usage. As the devices equipped with cameras and their associated applications grow in sophistication, the ways in which video streams can be utilized continues to broaden. The nascent field of wearable technology introduces the possibility for device control using gestures, detected by the built-in camera and processed by applications to accomplish a variety of tasks or commands previously clumsily accomplished using historically common means of input, such as a keyboard, pointing device, or voice control. The video feed from a wearable device, when coupled with a display positioned within the wearer's field of vision, also presents the opportunity for providing an augmented reality experience. Where the video stream coincides with the wearer's field of vision, an application can use image recognition techniques on the data stream to detect points of potential interest to the wearer, and provide notification of those points to the wearer by means of a display overlay.
Known implementations of managing data streams from hardware devices are not entirely satisfactory for the range of applications in which they are employed. For example, existing methods for managing the video stream from a camera require that an application be given exclusive control of the camera, effectively denying simultaneous access to the camera's data stream to any other applications that may need it. This is problematic when it is desirable for two applications that require the camera to be running simultaneously, such as in the foregoing example of one application that recognizes and acts upon user gestures made within the camera's field of view, and a second that interprets the camera stream and superimposes augmented reality cues on a transparent screen disposed within the user's field of view. Without simultaneous access to the camera data stream to both applications, it is impossible to present an augmented reality overlay while providing gesture recognition at the same time.
While this limitation could possibly be overcome using a single, monolithic application that provides both gesture recognition and the augmented reality overlay (as well as any other functionality that conceivably could be simultaneously desired), practical limits to the wearable device hardware platform must be recognized. A monolithic application typically imposes a greater memory footprint which, in the context of a wearable device or mobile platform that often has relatively limited working memory capacity when compared to a typical laptop or desktop computer, may result in fewer additional applications being able to run simultaneously. If separate processes for gesture recognition and augmented reality can be utilized, one or more of the applications can be unloaded when not needed (e.g. it may be desirable to have gesture recognition continuously active, but augmented reality overlays are only necessary at selected times), thereby saving working memory for other applications. Furthermore, a multiple process design is generally accepted as a more robust method of implementation as compared to a monolithic design, as a series of smaller modules are easier to debug, and any bugs that survive are isolated to a relatively limited functionality process that can be restarted.
Thus, there exists a need for methods of providing simultaneous access to a hardware data stream to multiple applications, improving upon and advancing the design of known hardware data stream access and sharing methods. Examples of new and useful methods for simultaneous hardware data stream access relevant to the needs existing in the field are discussed below.